Information processing apparatus, method, storage medium storing program, and system

ABSTRACT

An information processing apparatus sets a color mode in place of a monochrome mode on condition that the monochrome mode and multiplexing of additional information on a print target image are set as print settings based on an input image data; generates, based on the input image data, color image data corresponding to printing in a color mode which represents a color of the monochromated print target image by a value of a color signal; performs, for the color image data generated by the generation unit, processing for multiplexing the additional information on the print target image; and causes a printing apparatus to print, in the color mode, a multiplexed image on which the additional information is multiplexed.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an information processing apparatus formultiplexing additional information on image data, a method, a storagemedium storing a program, and a system.

Description of the Related Art

There is conventionally known a technique called an electronic watermarktechnique of multiplexing additional information on a print product insuch a way that it is difficult to visually discriminate (JapanesePatent Laid-Open No. 2003-174556). Superimposing data of another meaningon data, like steganography or watermark, or a technical field thereofwill generally be referred to as “multiplexing”. Multiplexing includesmultiplexing of identification information to prevent unauthorizedforgery of a banknote, a stamp, a security, or the like and multiplexingof audio information on a photo. When multiplexed information is readby, for example, a smartphone, it is possible to provide various addedvalue to a user.

SUMMARY OF THE INVENTION

The present invention provides an information processing apparatus forpreventing missing of multiplexed additional information due to printprocessing, a method, a storage medium storing a program, and a system.

The present invention in one aspect provides an information processingapparatus comprising: an acquisition unit configured to acquire inputimage data representing a print target image; a setting unit configuredto set a color mode in place of a monochrome mode on condition that themonochrome mode and multiplexing of additional information on the printtarget image are set as print settings based on the input image data; ageneration unit configured to generate, based on the input image data,color image data corresponding to printing in the color mode whichrepresents a color of the monochromated print target image by a value ofa color signal; a multiplexing unit configured to perform, for the colorimage data generated by the generation unit, processing for multiplexingthe additional information on the print target image; and a control unitconfigured to cause a printing apparatus to print a multiplexed image onwhich the additional information is multiplexed based on the color imagedata processed by the multiplexing unit in the color mode set by thesetting unit.

According to the present invention, it is possible to prevent missing ofmultiplexed additional information due to print processing.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing a system configuration including informationprocessing apparatuses;

FIG. 2 is a block diagram showing the hardware configuration of aninformation processing apparatus;

FIG. 3 is a block diagram showing an example of the hardwareconfiguration of an image processing apparatus;

FIGS. 4A and 4B are views showing masks;

FIGS. 5A and 5B are views visually showing patterns given to an image bythe masks;

FIG. 6 is a view showing a grayscale image;

FIG. 7 is a view showing a result of executing multiplexing;

FIG. 8 is a view showing a result of executing multiplexing;

FIG. 9 is a view showing a result of adapting FFT;

FIG. 10 is a view showing a result of adapting FFT;

FIG. 11 is a view showing a result of adapting FFT;

FIG. 12 is a view showing a result of adapting FFT;

FIG. 13 is a view showing a pattern for start position determination;

FIG. 14 is a flowchart showing processing to be executed by theinformation processing apparatus;

FIGS. 15A and 15B are views showing user interface screens;

FIG. 16 is a flowchart showing processing until print processing isstarted;

FIG. 17 is a view showing a dot image; and

FIG. 18 is a flowchart showing color conversion processing.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

If print settings are done to perform monochrome printing of image dataon which additional information is multiplexed by changing colorcomponents, the additional information is missing due to the processingof monochrome printing by a printer.

According to the present invention, it is possible to prevent missing ofmultiplexed additional information due to print processing.

FIG. 1 is a view showing an example of a system configuration includinginformation processing apparatuses 101 and 103 according to thisembodiment. The information processing apparatuses 101 and 103 areportable terminals such as a smartphone and a PC, and a description willbe made assuming that the information processing apparatuses 101 and 103have a common configuration. However, the configurations of theinformation processing apparatuses 101 and 103 may be different.

In the system, a user can store image data captured by an imagecapturing device inside. In the system, a router 104 and an imageprocessing apparatus 102, and the router 104 and the informationprocessing apparatus 103 are connected to be communicable with eachother by wired communication via a LAN cable or the like. Also, theinformation processing apparatus 101 and the router 104 are connected tobe communicable with each other by wireless communication such as Wifi.Note that the router 104 and the image processing apparatus 102 or theinformation processing apparatus 103 may be connected by wirelesscommunication. The wireless communication may be Bluetooth, NFC, or thelike in addition to Wifi.

In this embodiment, in the system as shown in FIG. 1 , the user canselect print target image data by an application on the informationprocessing apparatus 101 and multiplex arbitrary additional informationsuch as a watermark on the image data. That is, the informationprocessing apparatus 101 can operate as an additional informationmultiplexing apparatus that multiplexes additional information on imagedata. In addition, the user can print, using the image processingapparatus 102, the image data with the multiplexed additionalinformation. In this embodiment, text information will be described asan example of additional information. However, additional informationmay be information of another type, for example, audio information. Whenanother user captures the print product using the information processingapparatus 101, the multiplexed additional information is extracted. Thatis, the information processing apparatus 101 can operate as anadditional information extraction apparatus. For example, an operationof reproducing an audio by capturing a print product is implemented.

FIG. 2 is a block diagram showing an example of the hardwareconfiguration of the information processing apparatus 101. A CPU 201 isa central processing unit, and executes, for example, processing ofmultiplexing additional information in accordance with a program. A ROM202 stores programs to be executed by the CPU 201. A RAM 203 provides amemory used to temporarily store various kinds of information at thetime of program execution by the CPU 201. A secondary storage device 204is a storage medium configured to store an image file or additionalinformation to be multiplexed. A display unit 205 displays a userinterface screen or processing contents. The display unit 205 may have atouch panel function, and accepts an instruction of processing, asetting, character input, or the like in accordance with a useroperation on the touch panel. A speaker 206 outputs an audio. Forexample, if decoded and extracted additional information is audio data,the speaker 206 outputs an audio.

A network interface (IF) 207 is connected to a network 220 such as a LAN(Local Area Network). The network 220 may include the Internet. Forexample, the CPU 201 accesses, via the network interface 207, a siteprovided by a server 212 connected to the Internet and causes thedisplay unit 205 to display the screen of the site. An internal imagecapturing device 209 is a device having an image capturing function, andis, for example, an internal camera. An external image capturing device210 is an external device having an image capturing function andconnected to the information processing apparatus 101 via a deviceinterface (IF) 208, and is, for example, a digital camera or a videocamera.

In this embodiment, the information processing apparatus 101 serving asan additional information multiplexing apparatus will be described.However, the information processing apparatus 101 may operate as anadditional information multiplexing apparatus.

FIG. 3 is a block diagram showing an example of the hardwareconfiguration of the image processing apparatus 102. In this embodiment,the image processing apparatus 102 will be described as an MFP (MultiFunctional Peripheral). However, the image processing apparatus 102needs only to be a printing apparatus (printer) having a print functionand need not always have the configuration of an MFP. A control unit 301is connected to a scanner unit 311 that is an image input device and aprinter unit 312 that is an image output device, and controlsinput/output of image information to/from each device. In addition, thecontrol unit 301 can communicate with the information processingapparatus 101, the router 104, and the information processing apparatus103 via a network I/F 306. For example, the control unit 301 can receivea print job or image data from the information processing apparatus 101or the information processing apparatus 103.

A CPU 302 generally controls the operation of the image processingapparatus 102, and operates based on, for example, a program stored in aRAM 303. A ROM 304 is, for example, a boot ROM, and stores the bootprogram of the system. A storage unit 305 stores system software, imagedata, a program configured to control the operation of the imageprocessing apparatus 102, and the like. The CPU 302 loads the programstored in the storage unit 305 to the RAM 303 and controls the units ofthe image processing apparatus 102 based on the program.

The network I/F 306 connects the control unit 301 to the network 220,communicates with a device on the network 220, and inputs/outputsvarious kinds of information. A device I/F 307 connects the scanner unit311 and the printer unit 312, which are image input/output devices, tothe control unit 301, and performs conversion of the synchronoussystem/asynchronous system of image data. An operation unit I/F 308 isan interface that connects an operation unit 313 and the control unit301, and outputs, to the operation unit 313, data to be displayed on theoperation unit 313. The operation unit 313 includes a panel and hardwarekeys and can accept an instruction or a setting operation from the user.Also, the operation unit I/F 308 transmits, to the CPU 302, informationthat the user inputs from the operation unit 313.

An image processing unit 309 performs, for example, image processingsuch as correction, interpolation, and conversion according to a settingor the characteristic of an input/output device for print target imagedata received via the network. Also, the image processing unit 309performs image processing for image data to be input to the device I/F307 or output from the device I/F 307. An image memory 310 is a memoryconfigured to temporarily expand image data to be processed by the imageprocessing unit 309.

Multiplexing embedding processing (multiplexing encoding processing) tobe performed by the information processing apparatus 101 will bedescribed below. Here, a method of multiplexing information “hello” onan input image will be described as an example.

Handling information by an information processing apparatus such as a PCmeans handling binary data. Binary data is information “0” or “1”.Pieces of information “0” and “1” are continuously connected, therebyobtaining a specific meaning. For each character, corresponding binarydata is determined. Which character corresponds to which binary data isdefined by information called “character code”. For example, in “shiftJIS” that is one of character codes, “h” corresponds to binary data“01101000”. Similarly, “e” corresponds to binary data “01100101”, “1”corresponds to binary data “01101100”, and “o” corresponds to binarydata “01101111”.

That is, characters “hello” can be expressed by binary data“0110100001100101011011000110110001101111”. Reversely, if binary data“0110100001100101011011000110110001101111” can be acquired, characters“hello” can be acquired. That is, multiplexing can be implemented byembedding data such that “0” or “1” can be determined. In multiplexing,“hello” is input. However, actually embedded data is“0110100001100101011011000110110001101111”. In this embodiment, todiscriminate the two data, “hello” is called additional information(multiplexed information), and“0110100001100101011011000110110001101111” converted for embedding iscalled embedded data.

A method of multiplexing pieces of information “0” and “1” on an imagewill be described next. In this embodiment, embedding of datarepresenting “0” and “1” will be explained. However, the multiplexingmethod is not limited to embedding of “0” and “1”. The multiplexingmethod is not limited to embedding of “0” and “1” if it is possible touniquely associate data on the encoding side with data on the decodingside.

A description will be made assuming that the size of an image to bemultiplexed is 640 pixels (px) (height)×480 pixels (px) (width). Here,two masks shown in FIGS. 4A and 4B are prepared to generate “0” and “1”.

FIGS. 4A and 4B show masks each formed in a size of 8 px×8 px. Whenvalues in a mask are added to an image, a pattern having periodicity canbe given to an 8 px×8 px region in the image. FIGS. 5A and 5B visuallyshow what kinds of patters are given to an image by the masks. In FIGS.5A and 5B, a position of “10” in the masks shown in FIGS. 4A and 4B isexpressed by black, a position of “0” is expressed by gray, and aposition of “−10” is expressed by white. Oblique lines as shown in FIGS.5A and 5B appear in the image.

A pseudo code for alternately adapting the masks shown in FIGS. 4A and4B to an entire image is shown below.

01: int i, j, k, l; 02: int width = 640, height=480; 03: unsigned char*data = image data; 04: int **maskA = mask data; 05: bool isMaskA =true; 06:  for(j = 0; j < height; j+=8){ 07:   for(i = 0; i < width;i+=8){ 08:    for(k = 0; k < 8; k++){ 09:     for(l = 0; l < 8; l++){10:      if(isMaskA == true){ 11:       data[(i+k)+(j+l)*width] +=maskA[k][l]; 12: } 13: } 14: } 15: } 16: }

Image data in the third row assumes a grayscale image. Here, a grayscaleimage as shown in FIG. 6 is assumed. FIG. 7 shows an example in whichmultiplexing using the 8 px×8 px masks shown in FIGS. 4A and 4B isexecuted for the grayscale image shown in FIG. 6 . As shown in FIG. 7 ,oblique lines (corresponding to “0” and “1”) are alternately formed onthe image.

FIG. 8 shows an example in which“0110100001100101011011000110110001101111” that is data of “hello” isadapted, instead of simply repeating “0” and “1”. For “hello” itself,since 40 8 px×8 px blocks suffice, and a region having a size of 320px×8 px suffices. In FIG. 8 , “hello” is repetitively multiplexed. Whenthe same data is repetitively multiplexed in this way, resistance topattern disappearance caused by a flaw or dirt of a photo can beincreased. In addition, if a plurality of identical data exist, searchis easy in decoding.

In this example, since the image size is 640 px×480 px, the total numberof pixels is 307,200. Also, the number of pixels necessary formultiplexing “hello” is 320 px×8 px=2,560, as described above. That is,“hello” is repeated 307,200÷2560=120 times.

When multiplexing additional information, data embedding is performed,as described above. In general, additional information is multiplexed ina region to which a human eye is difficult to react. For example, a andb components after an image is decomposed into Lab components, U and Vcomponents after an image is decomposed into YUV components, or Cb andCr components of YCbCr are varied, thereby multiplexing additionalinformation. Also, for example, in the YUV components, a single Ucomponent may be varied, or both U and V components may be varied.

When varying a single component, the algorithm is simpler than in a casein which a plurality of components are varied. As a result, the programsize become small. For this reason, an application can be downloaded ata high speed, or reduction of the manufacturing cost can be implementedbecause the circuit area becomes small.

When a plurality of components are varied, it is possible to reduceinfluence that occurs due to the fact that data cannot be embedded. Forexample, if a digital image is expressed by 8 bits per color, one ofvalues 0 to 255 is assigned. Values outside the range cannot be used asimage data. For this reason, if the calculation result of a pixel valueis smaller than 0 or equal to or larger than 256, 0 or 255 is generallyassigned to make the value fall within the effective range. The masksshown in FIGS. 4A and 4B give a change of ±10 to each pixel value.However, for example, if all U components of YUV components in a maskregion have a value of 255, the values in the region fall within therange of not 245 to 265 but 245 to 255. In this case, only a variationsmaller than predetermined is given as a result, and multiplexing weakto noise and difficult to decode is performed. However, if the Ucomponents of YUV components have a value of 255, and V components havea value of 128, that is, if the pixel values can fall within theeffective range when changed by ±10, noise resistance can be maintainedby using the V components. As described above, when both the U and Vcomponents are used, the robustness of multiplexing (the fastness ofdecoding) can be increased. When converting an image into the RGBformat, the U and V components may be changed while confirming whetherRGB components after the YUV components are converted into RGBcomponents also fall within the range of 0 to 255.

As described above, when binary information is superimposed by giving aperiodical variation, embedding of data (multiplexing encoding) isimplemented.

In this embodiment, the information processing apparatus 101 may operateas an additional information extraction apparatus. Processing (decoding)of decoding embedded data in that case will be described.

The mask shown in FIG. 4A has a periodical pattern, and FFT (FastFourier Transform) is adapted to a grayscale image shown in FIG. 8 toextract the feature. FIG. 9 shows an example of a result of adapting FFTto an image in which data with a periodical pattern is embedded. On theother hand, FIG. 10 shows an example of a result of adapting FFT to thegrayscale image shown in FIG. 6 in which data embedding is notperformed. In FIG. 9 , it is found that spectra (white spots) appear atspecific positions, as compared to FIG. 10 . FIG. 11 is a view showingan example of a result of multiplexing only “0” on the grayscale imageshown in FIG. 6 and adapting FFT. FIG. 12 is a view showing an exampleof a result of multiplexing only “1” on the grayscale image shown inFIG. 6 and adapting FFT. FFT is a method used to analyze the frequencyof an image, and a variation of the periodical pattern of the maskchanges the frequency of the image. For this reason, when FFT isadapted, spectra appear at specific positions. Hence, when the state isread, embedded data can be recognized and decoded.

Even if both the characteristics of “0” and “1” are included, as shownin FIG. 9 , recognition needs to be possible. Hence, to determine one of“0” and “1”, FFT analysis is performed in the same size of 8 px×8 px asthe multiplexing block, thereby making the size in embedding match thesize in analysis. When FFT is executed only in this range, spectraappear in correspondence with only one value, and one of “0” and “1” canbe determined.

On the decoding side, when decoding data, the position where the datastarts may be specified. In this case, for example, a “pattern for startposition determination”, which is different from the pattern of “0” and“1”, is prepared, thereby enabling data start determination. FIG. 13 isa view showing an example of the pattern for start positiondetermination. As shown in FIG. 13 , the pattern for start positiondetermination is a mask pattern different from FIGS. 5A and 5B. In themask pattern shown in FIG. 13 , if a y-axis is defined along thevertical direction of the image, and an x-axis is defined along thehorizontal direction, the peak of spectra appears in the direction ofthe x-axis from the center of the image. If the pattern for startposition determination is detected at two points, additional informationcan be extracted by decoding “0” and “1” between the points.

Also, on the decoding side, processing for enabling FFT analysis may beperformed. If an image is acquired, via an image capturing sensor (forexample, a camera), from a print product on which additional informationis multiplexed, the image is not necessarily be obtained in a correctdirection. For example, the distance between the image capturing sensorand the print product may be short or long, and data may not always bedecoded even if the acquired image is FFT-analyzed for each 8 px×8 pxblock. In the above-described multiplexing, the image is changed suchthat a strong spectrum appears for a specific frequency, and where thespectrum should appear is known. Based on this, on the decoding side,image conversion may be performed before the FFT analysis such thatspectra obtained from the acquired image using the image capturingsensor become close to the known spectra.

The operation of the information processing apparatus 101 according tothis embodiment will be described below.

On the information processing apparatus 101, the user can do variouskinds of print settings for printing in the image processing apparatus102, and can set, for example, “monochrome printing” or “colorprinting”.

To multiplex additional information, variations are given to the colorcomponents of an image, as described above. On the other hand, if theuser sets “monochrome printing” as a print setting, processing(monochrome conversion) into two monochrome tones is performed in theimage processing apparatus 102. That is, even if additional informationis multiplexed, the color component on which multiplexing is performedis missing due to processing in two monochrome tones in the imageprocessing apparatus 102. As a result, the additional informationmultiplexed on the multiplexed image cannot be decoded. In thisembodiment, when multiplexing additional information, if the user sets“monochrome printing”, and the print target image data is color imagedata, grayscale conversion is performed, and setting conversion from“monochrome printing” to “color printing” is performed. With thisconfiguration, it is possible to maintain image quality close to thetint of monochrome printing desired by the user and prevent missing ofthe color component caused by performing processing in two monochrometones in the image processing apparatus 102.

FIG. 14 is a flowchart showing processing to be executed by theinformation processing apparatus 101 until printing of print targetimage data is executed. The processing shown in FIG. 14 is implementedby, for example, the CPU 201 reading out a program stored in the ROM 202to the RAM 203 and executing. The program configured to execute theprocessing shown in FIG. 14 is installed as, for example, a printapplication in the information processing apparatus 101. The processingshown in FIG. 14 is started when, for example, the user activates theprint application on the information processing apparatus 101.

In step S1401, the CPU 201 acquires selected image data (input imagedata) from the user. FIG. 15A is a view showing an example of a mainscreen 1501 displayed on the print application. As shown in FIG. 15A, abutton 1502 is displayed on the main screen 1501. When the button 1502is pressed by the user, an image selection screen (not shown) isdisplayed, and the use can select arbitrary image data. As an imageacquisition method, image data in the information processing apparatus101 may be acquired, or image data may be acquired from a server (notshown) via the network. Image data may be acquired from image dataprepared in advance in the print application. In this embodiment, imagedata will be described. However, not image data but document data suchas PDF may be used, and any data that can be converted intotwo-dimensional RGB data is usable. The image data selected as the printtarget by the user is image data of an additional informationmultiplexing target. The CPU 201 acquires the selected image data as themultiplexing target data, and temporarily stores it in the RAM 203 suchthat it can be reused in the print application.

Also, in a display region 1504 of the main screen 1501, the image of theimage data selected by the user is displayed. Note that if the storagearea of the RAM 203 in the information processing apparatus 101 is inshortage, the image data may be stored as a file in the secondarystorage device 204, and a reduced image (thumbnail image) obtained byreducing a designated image may be displayed on the screen of the printapplication.

In step S1402, the CPU 201 acquires additional information to bemultiplexed on the image data selected by the user in step S1401. Asshown in FIG. 15A, a check box 1503 and an input region 1505 areprovided in the main screen 1501. The user can instruct multiplexing onthe selected image data by checking the check box 1503. Also, the usercan input arbitrary text information to the input region 1505. The CPU201 acquires the text information input to the input region 1505 asadditional information to be multiplexed on the image data.

In this embodiment, text information will be described as multiplexinginformation. However, this may be information of another type such as anaudio, a photo, or a moving image, which can be handled as binary data.The CPU 201 stores the text information input by the user in the RAM 203such that it can be reused in the print application or as a file in thesecondary storage device 204.

In step S1403, the CPU 201 searches for the image processing apparatus102. As shown in FIG. 15A, a button 1506 is displayed in the main screen1501. If the button 1506 is pressed by the user, the CPU 201 transmits aprinter search broadcast to the network 220. By the broadcast,information is transmitted to all hosts that share the same network.When the image processing apparatus 102 returns a reply to theinformation processing apparatus 101 that is the broadcast transmissionsource, the information processing apparatus 101 can grasp the IPaddress of the image processing apparatus 102, and one-to-onecommunication can be performed from then on. In this embodiment, theidentification information of a printer that has responded isautomatically registered in the print application. In this example, forthe descriptive convenience, the printer in the network 220 is assumedto be only the image processing apparatus 102. Pieces of informationregistered in the print application are, for example, a printer name andthe IP address of the printer. The printer name is displayed as aprinter name 1507 on the main screen 1501.

In step S1404, the CPU 201 acquires information concerning printsettings from the image processing apparatus 102 found in step S1403,and accepts print settings from the user. Normally, the printer canacquire a medium (print medium), size, print settings, and the like thatthe printer itself supports.

In this embodiment, communication between the print application in theinformation processing apparatus 101 and the image processing apparatus102 is done by XML (Extensible Markup Language). XML is one of methodsof expressing information. Data transmission/reception is notnecessarily performed by this, and another protocol or communicationmethod may be used. An example of XML for requesting the imageprocessing apparatus 102 to acquire information concerning printsettings is shown below.

001:<?xml version=“1.0” encoding=“UTF-8”?> 002:<cmd>003:<operation>getInformation</operation> 004:</cmd>

The first row indicates that the XML is version 1.0, and the characterencoding scheme is UTF-8. <cmd> in the second row means the start of acommand received by the printer, and </cmd> in the fourth row indicatesthe end of the command. When a tag is added in the cmd tag andtransmitted to the image processing apparatus 102 by the printapplication, the image processing apparatus 102 executes an operationaccording to the contents of the tag. For example, operation in thethird row describes the operation type of the printer, andgetInformation represents an operation of returning acquired informationto the application of the request source. An example of XML that theimage processing apparatus 102 returns to the print application is shownbelow.

001:<?xml version=“1.0” encoding=“UTF-8”?> 002:<res>003:<name>printer001</name> 004:<medialist> 005: <media>006:  <name>A4</name> 007:  <width>2000</width>008:  <height>3000</height> 009: </media> 010: <media>011:  <name>L</name> 012:  <width>1000</width>013:  <height>1500</height> 014: </media> 015: <media>016:  <name>Hagaki</name> 017:  <width>1500</width>018:  <height>2250</height> 019: </media> 020: </medialist>021:  <colormode> 022:   <mode>color</mode> 023:   <mode>mono</mode>024:  </colormode> 025: </res>

res in the second row means a response from the printer. In thisembodiment, a command starting from <cmd> is used for information fromthe print application to the printer, and a command starting from <res>is used for information from the printer to the print application.However, the present invention is not limited to this. name in the thirdrow indicates the printer name. The fourth to 20th rows represent thelist of supported paper sizes, and media that the printer can supportare listed in the <medialist> tag. Here, in the <media> tag, tags ofname, width, and height exist, which represent the name of a sheet, animage height (pixel), and an image width (pixel), respectively. Based onthese pieces of information, it can be seen that if the paper size is,for example, A4 (210 mm (vertical)×297 mm (horizontal)), an image sizeof 2,000 px (width)×3,000 px (height) is necessary. In general, theimage size depends on dpi (dots per inch) of the printer. dpi is a unitrepresenting how finely ink can be arranged. If the value of dpi ishigh, a high-definition expression, that is, so-called high-resolutionprinting is possible. That is, even in the same A4 size, the higher thedpi is, the larger the number of pixels of image data needed forprinting is. On the other hand, image data designated by the printapplication does not always have the number of pixels necessary for theprinter. In this case, enlargement or reduction, or trimming isperformed in the printer or by a printer driver to correct the inputimage into a size necessary for the printer, and printing is performedafter that. For example, when printing an image of 4,000 px(height)×6,000 px (width) using the image processing apparatus 102, itis impossible to directly execute print processing because the dataincludes pixels in number more than necessary for the A4 size. Hence,the image data is converted into, for example, 2,000 px (height)×3,000px (width) in the printer or by the printer driver, and printing isperformed after that.

In the 21st to 24th rows, selectable print color setting modes arelisted. In this embodiment, two types of modes, that is, “color” that isa color mode and “mono” that is a monochrome mode can be selected. Thecolor mode is a print mode in which printing is performed using aplurality of colors of inks of the printer. The printer normallyexpresses colors by subtractive color mixture. In subtractive colormixture, normally, all colors can be expressed by three colors, that is,C (cyan), M (magenta), and Y (yellow), and the image processingapparatus 102 uses inks of three colors C, M, and Y. Note that the inktypes are not limited to these, and gray ink that is a spot ink or inkcolors such as RGB other than CMY may be provided.

On the other hand, the monochrome mode is a print mode in which printingis performed after the color information of an input image is replacedwith gray information. The monochrome mode is sometimes performed inorder to, for example, decrease the ink amount to be used or improve theprinting speed. In the monochrome mode, a single black ink (K) or aplurality of achromatic color inks are used. Also, process black may beformed by CMYK. Note that in this embodiment, a mode for implementingmonotone colors, such as a sepia mode or a grayscale mode, is includedin the monochrome mode.

In this embodiment, to multiplex additional information in a case inwhich the monochrome mode using only black ink is set by the user, theprint setting is changed from “monochrome mode” to “color mode”. As aresult, in the image processing apparatus 102, color printing isperformed using color inks. Even if the print setting is changed to“color mode”, the tint of an output product is close to gray from theviewpoint of image quality expected in the monochrome mode. That is, animage whose tone is expressed by the density of dots printed in black isoutput.

FIG. 15B is a view showing an example of a print setting screen 1510displayed on the print application. The print setting screen 1510 isdisplayed when a print setting button 1508 on the main screen 1501 ispressed. The print setting screen 1510 is displayed based on informationconcerning the print settings acquired from the image processingapparatus 102. As shown in FIG. 15B, the print setting screen 1510 isdisplayed such that selection of the items of a paper size 1511 and aprint color mode 1513 can be accepted.

Here, assume that the user turns on the check box 1503 for executingmultiplexing processing, and sets a paper size “A4” indicated by aselection button 1512 and a print color mode “monochrome” indicated bythe selection button 1513. Upon accepting pressing of a decision button1514 from the user, the CPU 201 temporarily stores the items that theuser sets on the print setting screen 1510 in a storage area of the RAM203 or the like such that these can be referred to later. When thedecision button 1514 is pressed, display returns to the display of themain screen 1501. After pressing of a print button 1509 on the mainscreen 1501 is accepted from the user, print processing by the imageprocessing apparatus 102 is started.

FIG. 16 is a flowchart showing processing until print processing by theimage processing apparatus 102 is started. The processing shown in FIG.16 is implemented by, for example, the CPU 201 reading out a programstored in the ROM 202 to the RAM 203 and executing. Also, on conditionthat the check box 1503 and the selection button 1513 are set asdescribed above, the processing shown in FIG. 16 is started when theprint button 1509 on the main screen 1501 is pressed.

In step S1601, the CPU 201 acquires, from the RAM 203, print targetimage data selected by the user on the main screen 1501. In step S1602,the CPU 201 acquires additional information input by the user on themain screen 1501, for example, text data. In step S1603, the CPU 201acquires print settings set by the user.

In step S1604, the CPU 201 determines, based on the print settingsacquired in step S1603, whether the monochrome mode is set. Upondetermining that the monochrome mode is set, in step S1605, the CPU 201changes the print setting from “monochrome mode” to “color mode”. Asdescribed above, if the monochrome mode is set as the print setting,monochrome conversion is executed in the printer or by the printerdriver. For example, when executing monochrome conversion in theprinter, image data that has undergone multiplexing is transmitted tothe image processing apparatus 102, and monochrome conversion is thenexecuted in the printer. Here, monochrome conversion is conversion forsetting one color serving as a reference in an image and express theimage only by the shading change of the color. As image data, this isexpressed as the density of data in a certain region. That is,components other than luminance component, for example, a and b in Labor U and V in YUV do not change. In the method of executing multiplexingby giving variations to components other than luminance, additionalinformation is missing, and decoding cannot be performed. Hence, in thisembodiment, if the monochrome mode is set by the user, the print settingis changed to the color mode such that monochrome conversion is notexecuted in the printer or by the printer driver. At this time,predetermined color conversion processing is executed such that a tintcorresponding to the monochrome mode desired by the user can be outputeven in the color mode. With this configuration, to the user, the imageprocessing apparatus appears to operate in accordance with the set printmode and additional information multiplexing processing is possible. Inthis embodiment, as color conversion processing that is an alternate tomonochrome conversion using black ink, grayscale conversion is executedas the predetermined color conversion processing.

For branch of processing from step S1604, for example, the followingconfiguration may be used. For example, a black list may be held inadvance in the secondary storage device 204 or a program, and the blacklist may be compared with the information concerning the print settingacquired from the image processing apparatus 102. The black list is alist that describes the names of services and software that should notbe used. For example, a character string “mono” is described as a textfile in the black list. If the print setting matches the description ofthe black list, the process advances to step S1606. Note that to hold aplurality of pieces of information in the black list or describe colorconversion processing to be executed if the description matches theblack list, CSV (Comma-Separated Values) or a general database may beused.

Alternatively, the print application may acquire information from theserver on the network. The print application can acquire the printername by communicating with the image processing apparatus 102. A querymay be transmitted to the server using the printer name, and branch ofprocessing from step S1604 may be performed. In this case, the serverholds the printer name and the print setting, and informationrepresenting whether to perform monochrome conversion for thecombination. That is, in this configuration, the black list is held onthe network side.

In addition, information acquired from the image processing apparatus102 may be used. Information representing whether multiplexing ispossible may be added to colormode in XML, including the above-describedprinter information. For example, the following XML is used.

001:<?xml version=“1.0” encoding=“UTF-8”?> 002:<res> 003:~ partiallyomitted ~ 004:<colormode> 005: <mode> 006:  <modename>color</modename>007:  <steganography>true</steganography > 008: </mode> 009: <mode>010:  <modename>mono</modename>011:  <steganography>false</steganography >012:  <alternate>grayscaletransform</alternate>013:  <colormode>color</colormode> 014: </mode> 015:</colormode>016:</res>

In the above XML, new information is added to <mode> of <colormode>, ascompared to the above-described XML. First, <steganography> in the 11throw includes information representing whether multiplexed additionalinformation is missing due to the color mode setting. If this tag isfalse, this means that the multiplexed additional information ismissing. The <alternate> tag in the 12th row describes an alternatemethod for preventing additional information from missing, and<colormode> in the 13th row describes a print setting after thealternate method is used. The above notation represents information“multiplexing cannot directly be performed in the monochrome mode, butinstead, grayscale conversion is adapted to the image, and the printcolor mode is set to color”. For example, if the information acquiredfrom the image processing apparatus 102 represents the above-describedinformation, processing after step S1605 is performed. Ingrayscaletransform in the <alternate> tag, corresponding processing onthe print application side is grasped. On the other hand, the contentsof processing itself may be described. For example, a program for imageconversion may directly be written in the XML in the following way,interpreted on the print application side, and executed.

~ omitted ~ 001: <alternate> 002: function transform(r, g, b){ 003:  vargray = 0.299*r + 0.587*g + 0.114*b; 004:  return {r: gray, g:gray,b:gray}; 005: } 006: </alternate> ~ omitted ~

The second to fifth rows represent a function according to theJavaScript notation. This is a function for returning R, G, and B afterchange when the pixel data of R (Red), G (Green), and B (Blue) are givenas arguments. The third row corresponds to processing ofgrayscale-converting R, G, and B. When this function is acquired by theapplication, and the result of interpreting this by an interpreteroperating on the CPU 201 is executed from the application, dynamicprogram execution is possible. For example, when an eval functionprovided by JavaScript is used, the above-described function can beinterpreted as a program and executed from the application. Also,another script language may be used, or an interpreter that interpretsprocessing contents and executes may be prepared independently. When anotation as described above is used, even if a print setting other thanmonochrome conversion that causes missing of multiplexed additionalinformation is added to the image processing apparatus 102, it ispossible to cope with this as needed.

In this embodiment, a case in which the print application holds theblack list and copes with only a monochrome print setting will bedescribed as an example. The black list may be incorporated in the printapplication (stored in the secondary storage device 204), or may bedirectly described as a program.

If the print setting is changed from “monochrome mode” to “color mode”in step S1605, in step S1606, the CPU 201 executes grayscale conversionfor the multiplexing target image data. In monochrome conversion,normally, printing using one color ink is performed, and a tone isexpressed by shading. In this embodiment, however, grayscale conversionis performed as an alternate to monochrome conversion using black ink.That is, multiplexing processing to be executed in step S1609 of thesubsequent stage is performed based on grayscale-converted image data.For example, multiplexing processing to be executed in step S1609 of thesubsequent stage may be performed for grayscale-converted image datahaving gray tones of 0 to 255, or may be performed for a halftone imagewhose pixel values are expressed only by the density of two values of255 (white) and 0 (black). However, in this embodiment, thegrayscale-converted image data or the image data expressed only by thedensity of two values is converted into image data of a plurality ofchannels such as RGB, and as a result, multiplexing using variations ofcolor components is possible.

FIG. 17 is a view showing an example of an image 1701 expressed by thedensity of two values. An image 1703 is an enlarged image of a part 1702of the image 1701. It can be found that the density of black and whitechanges depending on a region. Here, for example, a mask 1704 of 4 px×4px is used. The mask 1704 is arranged on the image region shown in FIG.6 without overlapping, and a value of the mask 1704 and a pixel value ofthe image shown in FIG. 6 are compared. If the pixel value is smallerthan the value of the mask 1704, the pixel value is converted into apixel value of 0 (black). If the pixel value is larger, the pixel valueis converted into a pixel value of 255 (white). Note that although thepixel value of 255 is expressed as white on image data, the imageprocessing apparatus 102 does not discharge ink to the region of thepixel value of 255 if the print medium is white.

FIG. 18 is a flowchart showing an example of the processing of stepS1606. In step S1801, the CPU 201 determines whether the image datainput to the processing of step S1606 is of a format of a plurality ofchannels, for example, whether the image data is formed by a pluralityof channels such as RGB or YCbCr. Upon determining that the image datais formed by a plurality of channels, in step S1802, the CPU 201converts it into image data formed by a single channel (grayscaleconversion). In step S1802, for example, RGB image data is convertedinto grayscale image data expressed by monochromating or Y of YCbCr. Instep S1803, the CPU 201 executes halftone processing by a dither methodor an error diffusion method. On the other hand, upon determining instep S1801 that the image data is image data formed by a single channel,the halftone processing of step S1803 is executed after that.

After step S1803, the CPU 201 converts the image data generated in stepS1803 into data of a plurality of channels represented by the values ofcolor signals. For example, the image data generated in step S1803 isconverted into RGB image data corresponding to black (pixel value “0”)and white (pixel value “255”). When the image data of a plurality ofchannels is generated in this way, additional information multiplexingto be executed in the subsequent stage by varying color components ispossible. After step S1804, the processing shown in FIG. 18 is ended.

When the processing shown in FIG. 18 is performed, input image data canbe converted into image data expressed only by the density of two values(black and white). The size of an ink dot printed by the imageprocessing apparatus 102 is much smaller than the resolving power of ahuman eye. For this reason, the actual print product is recognized bythe human eye not as the density of dots, like the image 1703, but asshading of black and white.

After step S1606, the process advances to step S1607. Upon determiningin step S1604 that “color mode” is set, processing of step S1607 isexecuted. In step S1607, the CPU 201 determines whether the image sizeof the image data matches the size (for example, A4) necessary forprinting. Upon determining that the image size does not match the sizenecessary for printing, in step S1608, the CPU 201 executes resizeprocessing to perform resize to the size necessary for printing. This isbecause if resize of the image is performed by the printer driver or inthe printer, the period of spectra intended in the multiplexingembedding changes and affects decoding. In particular, if reduction isexecuted in resize, multiplexed additional information may be missingbecause pixels are thinned. In this embodiment, resize of the image isperformed before multiplexing processing, thereby preventing multiplexedadditional information from missing due to resize of the image. As theresize, linear interpolation such as nearest neighbor, bilinear, orbicubic may be used. Also, in this embodiment, grayscale conversion isperformed, and after that, resize is performed. These processes may beperformed in a reverse order. After step S1608, or upon determining instep S1607 that the image size matches the size necessary for printing,the process advances to step S1609.

In step S1609, the CPU 201 multiplexes the additional information, forexample, text data acquired in step S1603 on the image data processed upto step S1607 or S1608. The multiplexing processing is executed forcolor components, as described above, for example, CbCr components afterconversion from RGB to YCbCr. When multiplexing processing is executedby the processing shown in FIG. 16 , it is possible to print the imagedata in a decodable state by the image processing apparatus 102 whileguaranteeing the print settings intended by the user.

In step S1610, the CPU 201 transmits a printing execution instruction tothe image processing apparatus 102. An example of XML, for instructingprinting execution is shown below.

001: <?xml version=“1.0” encoding=“UTF-8”?> 002: <cmd> 003:<operation>print</operation> 004: <data>base64 data of jpeg</data > 005:</cmd>

print of the <operation> tag in the third row indicates printingexecution. The <data> tag in the fourth row describes image data. Here,it is described, as an example, that a jpeg image of the base64 formatis transmitted.

After step S1610, the processing shown in FIG. 16 is ended. After theprocessing shown in FIG. 16 is ended, the image processing apparatus 102that has received the transmitted information starts printing.

As described above, in this embodiment, if a print setting that maycause missing of multiplexed additional information is done, processingthat is an alternate to processing to be performed under the printsetting is performed, and then, multiplexing of additional informationis executed. Hence, it is possible to prevent multiplexed additionalinformation from missing and perform printing in a decodable state whilemaintaining image quality desired by the user.

Also, in this embodiment, the processing shown in FIGS. 14 and 16 isexecuted by the information processing apparatus 101. However, theprocessing may be executed by the image processing apparatus 102. Inthis case, for example, image data captured by the informationprocessing apparatus 101 such as a smartphone is acquired as image dataof the print target and multiplexing target.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2021-097465, filed Jun. 10, 2021, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An information processing apparatus comprising:an acquisition unit configured to acquire input image data representinga print target image; a setting unit configured to set a color mode inplace of a monochrome mode on condition that the monochrome mode andmultiplexing of additional information on the print target image are setas print settings based on the input image data; a generation unitconfigured to generate, based on the input image data, color image datacorresponding to printing in the color mode which represents a color ofthe monochromated print target image by a value of a color signal; amultiplexing unit configured to perform, for the color image datagenerated by the generation unit, processing for multiplexing theadditional information on the print target image; and a control unitconfigured to cause a printing apparatus to print a multiplexed image onwhich the additional information is multiplexed based on the color imagedata processed by the multiplexing unit in the color mode set by thesetting unit.
 2. The apparatus according to claim 1, further comprisinga first conversion unit configured to convert the input image data intoimage data of a single channel if the input image data is image data ofa plurality of channels, wherein the generation unit generates the colorimage data based on the image data of the single channel converted bythe first conversion unit.
 3. The apparatus according to claim 2,further comprising a second conversion unit configured to convert theimage data of the single channel into color image data of a plurality ofchannels, wherein the color image data as a target of the multiplexingunit is the color image data of the plurality of channels.
 4. Theapparatus according to claim 3, wherein the multiplexing unitmultiplexes the additional information by varying a color component. 5.The apparatus according to claim 1, further comprising a firstdetermination unit configured to determine whether to execute setting ofthe color mode by the setting unit, wherein if it is determined, by thefirst determination unit, to execute the setting of the color mode bythe setting unit, the setting unit sets the color mode in place of themonochrome mode.
 6. The apparatus according to claim 5, wherein thefirst determination unit determines to execute the setting of the colormode by the setting unit based on execution of processing correspondingto the monochrome mode in the printing apparatus.
 7. The apparatusaccording to claim 6, wherein if a setting by a user includes themonochrome mode, the first determination unit determines to execute thesetting of the color mode by the setting unit.
 8. The apparatusaccording to claim 6, wherein upon determining, based on informationacquired from the printing apparatus, that the processing correspondingto the monochrome mode is executed in the printing apparatus, the firstdetermination unit determines to execute the setting of the color modeby the setting unit.
 9. The apparatus according to claim 8, wherein thefirst determination unit determines, based on the information acquiredfrom the printing apparatus and a black list, whether the processingcorresponding to the monochrome mode is executed in the printingapparatus.
 10. The apparatus according to claim 1, further comprising asecond determination unit configured to determine, based on a size of aprint medium on which printing based on the input image data is to beexecuted, whether to execute resize of the color image data, wherein ifit is determined, by the second determination unit, to execute theresize of the color image data, the resize of the color image data isexecuted before the multiplexing of the additional information by themultiplexing unit.
 11. The apparatus according to claim 1, wherein ifthe color mode is set as the print setting based on the input imagedata, the additional information is multiplexed on the input image datawithout performing setting of the color mode by the setting unit. 12.The apparatus according to claim 1, further comprising a display unitconfigured to display a user interface screen, wherein the input imagedata acquired by the acquisition unit is image data selected by a useron the user interface screen.
 13. The apparatus according to claim 12,wherein the additional information is information input by the user onthe user interface screen.
 14. The apparatus according to claim 1,further comprising the printing apparatus.
 15. A method comprising:acquiring input image data representing a print target image; setting acolor mode in place of a monochrome mode on condition that themonochrome mode and multiplexing of additional information on the printtarget image are set as print settings based on the input image data;generating, based on the input image data, color image datacorresponding to printing in the color mode which represents a color ofthe monochromated print target image by a value of a color signal;performing, for the generated color image data, processing formultiplexing the additional information on the print target image; andcausing a printing apparatus to print a multiplexed image on which theadditional information is multiplexed based on the color image dataprocessed in the multiplexing in the set color mode.
 16. Anon-transitory computer-readable storage medium storing a programconfigured to cause a computer to operate to: acquire input image datarepresenting a print target image; set a color mode in place of amonochrome mode on condition that the monochrome mode and multiplexingof additional information on the print target image are set as printsettings based on the input image data; generate, based on the inputimage data, color image data corresponding to printing in the color modewhich represents a color of the monochromated print target image by avalue of a color signal; perform, for the generated color image data,processing for multiplexing the additional information on the printtarget image; and cause a printing apparatus to print a multiplexedimage on which the additional information is multiplexed based on thecolor image data processed in the multiplexing in the set color mode.17. A system including an information processing apparatus and aprinting apparatus, the information processing apparatus comprising: anacquisition unit configured to acquire input image data representing aprint target image; a setting unit configured to set a color mode inplace of a monochrome mode on condition that the monochrome mode andmultiplexing of additional information on the print target image are setas print settings based on the input image data; a generation unitconfigured to generate, based on the input image data, color image datacorresponding to printing in the color mode which represents a color ofthe monochromated print target image by a value of a color signal; amultiplexing unit configured to perform, for the color image datagenerated by the generation unit, processing for multiplexing theadditional information on the print target image; and a transmissionunit configured to transmit, to the printing apparatus, image data onwhich the additional information is multiplexed based on the color imagedata processed by the multiplexing unit in the color mode set by thesetting unit, and the printing apparatus printing a multiplexed image onwhich the additional information is multiplexed based on the image datatransmitted by the transmission unit.